我们研究马尔可夫游戏子类的策略梯度方法的性能，称为马尔可夫潜在游戏（MPGS），该游戏将正常形式潜在游戏的概念扩展到了状态环境，其中包括完全合作环境的重要特殊情况。代理商具有相同的奖励功能。我们本文的重点是研究在SoftMax策略参数化下求解MPG的策略梯度方法的收敛性，无论是表格和参数，都用一般函数近似器（例如神经网络）进行参数化。我们首先显示了该方法对MPG的NASH平衡的渐近收敛性，以进行表格软智能策略。其次，我们在两个设置中得出了策略梯度的有限时间性能：1）使用对数屏障正则化，以及2）在最佳反应动力学（NPG-BR）下使用自然策略梯度。最后，我们在正常游戏中扩展了无政府状态（POA）的价格（POA）的概念，我们介绍了MPG的POA，并为NPG-BR提供了POA。据我们所知，这是第一个用于解决MPG的POA。为了支持我们的理论结果，我们从经验上比较了表格和神经软性策略的策略梯度变体的收敛速率和POA。

我们研究了在两人零和马尔可夫游戏中找到NASH平衡的问题。由于其作为最小值优化程序的表述，解决该问题的自然方法是以交替的方式对每个玩家进行梯度下降/上升。但是，由于基本目标函数的非跨性别/非障碍性，该方法的理论理解是有限的。在我们的论文中，我们考虑解决马尔可夫游戏的熵登记变体。正则化将结构引入了优化景观中，从而使解决方案更加可识别，并允许更有效地解决问题。我们的主要贡献是表明，在正则化参数的正确选择下，梯度下降算法会收敛到原始未注册问题的NASH平衡。我们明确表征了我们算法的最后一个迭代的有限时间性能，该算法的梯度下降上升算法的现有收敛界限大大改善了而没有正则化。最后，我们通过数值模拟来补充分析，以说明算法的加速收敛性。

由于其许多领域的广泛应用程序，包括机器学习，网络资源分配和分布式优化，因此在解决非协议敏最大优化问题中有很多兴趣。也许，求解最小最大优化的最受欢迎的一阶方法是所谓的同时（或单环）梯度下降 - 上升 - 上升算法，因为它的实施简单。然而，对该算法的收敛性的理论保证非常稀疏，因为即使在简单的双线性问题中也可以发散。在本文中，我们的重点是表征同时梯度下降算法的连续时间变量的有限时间性能（或收敛速率）。特别是，我们在底层目标函数的许多不同条件下得出了这种方法的收敛速度，即双面Polyak-L OjasiewiCz（PL），单侧PL，非凸起强烈凹入，强烈凸-Nonconcave条件。我们的趋同结果在目标职能的相同条件下提高了先前作品中的结果。我们分析中的关键思路是使用经典奇异扰动理论和耦合Lyapunov函数来解决梯度下降和上升动力学之间的时间尺度差异和相互作用。我们对连续时间算法行为的结果可用于增强其离散时间对应的收敛性。

我们考虑了折现成本约束的马尔可夫决策过程（CMDP）策略优化问题，其中代理商试图最大化折扣累计奖励，但受到折扣累积公用事业的许多限制。为了解决这个受约束的优化程序，我们研究了经典原始偶性方法的在线参与者 - 批判性变体，其中使用来自基本时间变化的马尔可夫过程产生的单个轨迹的样品估算了原始功能和双重函数的梯度。这种在线原始双重自然参与者批评算法维护并迭代更新三个变量：双变量（或拉格朗日乘数），一个原始变量（或actor）以及用于估算原始变量和偶变量的梯度的评论变量。这些变量同时更新，但在不同的时间尺度上（使用不同的步骤尺寸），它们都相互交织在一起。我们的主要贡献是得出该算法与CMDP问题全局最佳收敛的有限时间分析。具体而言，我们表明，在适当的步骤中，最佳差距和约束违规的情况下，以$ \ mathcal {o}（1/k^{1/6}）$的价格收敛到零，其中k是数字。迭代。据我们所知，本文是第一个研究用于解决CMDP问题的在线原始偶发参与者方法的有限时间复杂性。我们还通过数值模拟来验证该算法的有效性。

我们研究了随机近似的分散变体，这是一种数据驱动的方法，用于在嘈杂的测量中找到操作员的根。一个具有自己的操作员和数据观察的代理网络，合作地通过分散的通信图找到了聚合操作员的固定点。我们的主要贡献是在从马尔可夫过程中采样时在每个代理下观察到的数据时，对这种分散的随机近似方法提供有限的时间分析；这种缺乏独立性使迭代率偏向和（可能）无限。在相当标准的假设下，我们表明所提出方法的收敛速率与样本是独立的基本相同，仅由对数因子的差异而不同，该对数因素是说明了马尔可夫过程的混合时间。我们的分析中的关键思想是引入一种新型的Razumikhin-Lyapunov函数，该功能是由用于分析延迟普通微分方程的稳定性的一种动机。我们还讨论了拟议方法在多代理系统中许多有趣的学习问题上的应用。

The sentiment analysis task has various applications in practice. In the sentiment analysis task, words and phrases that represent positive and negative emotions are important. Finding out the words that represent the emotion from the text can improve the performance of the classification models for the sentiment analysis task. In this paper, we propose a methodology that combines the emotion lexicon with the classification model to enhance the accuracy of the models. Our experimental results show that the emotion lexicon combined with the classification model improves the performance of models.

While the capabilities of autonomous systems have been steadily improving in recent years, these systems still struggle to rapidly explore previously unknown environments without the aid of GPS-assisted navigation. The DARPA Subterranean (SubT) Challenge aimed to fast track the development of autonomous exploration systems by evaluating their performance in real-world underground search-and-rescue scenarios. Subterranean environments present a plethora of challenges for robotic systems, such as limited communications, complex topology, visually-degraded sensing, and harsh terrain. The presented solution enables long-term autonomy with minimal human supervision by combining a powerful and independent single-agent autonomy stack, with higher level mission management operating over a flexible mesh network. The autonomy suite deployed on quadruped and wheeled robots was fully independent, freeing the human supervision to loosely supervise the mission and make high-impact strategic decisions. We also discuss lessons learned from fielding our system at the SubT Final Event, relating to vehicle versatility, system adaptability, and re-configurable communications.

We present Muse, a text-to-image Transformer model that achieves state-of-the-art image generation performance while being significantly more efficient than diffusion or autoregressive models. Muse is trained on a masked modeling task in discrete token space: given the text embedding extracted from a pre-trained large language model (LLM), Muse is trained to predict randomly masked image tokens. Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding. The use of a pre-trained LLM enables fine-grained language understanding, translating to high-fidelity image generation and the understanding of visual concepts such as objects, their spatial relationships, pose, cardinality etc. Our 900M parameter model achieves a new SOTA on CC3M, with an FID score of 6.06. The Muse 3B parameter model achieves an FID of 7.88 on zero-shot COCO evaluation, along with a CLIP score of 0.32. Muse also directly enables a number of image editing applications without the need to fine-tune or invert the model: inpainting, outpainting, and mask-free editing. More results are available at https://muse-model.github.io

Optical coherence tomography (OCT) captures cross-sectional data and is used for the screening, monitoring, and treatment planning of retinal diseases. Technological developments to increase the speed of acquisition often results in systems with a narrower spectral bandwidth, and hence a lower axial resolution. Traditionally, image-processing-based techniques have been utilized to reconstruct subsampled OCT data and more recently, deep-learning-based methods have been explored. In this study, we simulate reduced axial scan (A-scan) resolution by Gaussian windowing in the spectral domain and investigate the use of a learning-based approach for image feature reconstruction. In anticipation of the reduced resolution that accompanies wide-field OCT systems, we build upon super-resolution techniques to explore methods to better aid clinicians in their decision-making to improve patient outcomes, by reconstructing lost features using a pixel-to-pixel approach with an altered super-resolution generative adversarial network (SRGAN) architecture.

Compliance in actuation has been exploited to generate highly dynamic maneuvers such as throwing that take advantage of the potential energy stored in joint springs. However, the energy storage and release could not be well-timed yet. On the contrary, for multi-link systems, the natural system dynamics might even work against the actual goal. With the introduction of variable stiffness actuators, this problem has been partially addressed. With a suitable optimal control strategy, the approximate decoupling of the motor from the link can be achieved to maximize the energy transfer into the distal link prior to launch. However, such continuous stiffness variation is complex and typically leads to oscillatory swing-up motions instead of clear launch sequences. To circumvent this issue, we investigate decoupling for speed maximization with a dedicated novel actuator concept denoted Bi-Stiffness Actuation. With this, it is possible to fully decouple the link from the joint mechanism by a switch-and-hold clutch and simultaneously keep the elastic energy stored. We show that with this novel paradigm, it is not only possible to reach the same optimal performance as with power-equivalent variable stiffness actuation, but even directly control the energy transfer timing. This is a major step forward compared to previous optimal control approaches, which rely on optimizing the full time-series control input.